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INTRODUCTION  The  analysis  of  cross-classified  categorical  data 
Involves  statistical  problems  where  both  the  explanatory  variables 
(or  factors)  and  response  variables  are  categorical.  Logllnear  and 
logit  models  are  now  wldely-uacd  in  the  analysis  of  such  data  (e.g. 
sec  Bishop,  Flenberg,  and  Holland,  1975;  Flenberg  1977;  Haberman, 
1974,  1978).  Since  logit  models  are  logllnear  models,  the 
computational  methods  for  the  analysis  of  cross-classifications  via 
logllnear  models  can  also  be  used  for  analyses  Involving  the  use  of 
logit  models. 


In  this  paper  we  compare  three  different  computational  approaches 
for  maximum  likelihood  estimation  In  logit  situations: 

(a)  Iterative  proportional  fitting, 

(b)  Iteratively  rewelghted  least  squares  as  Implemented 
In  GLIM  (see  Nelder  and  Wedderbum,  1972), 

(c)  a variant  of  Newton's  method,  as  developed  by 
Flenberg  and  Stewart  (1979),  applied  In  a somewhat 
different  form  for  logllnear  and  logit  formulations. 

Additional  comparisons  can  be  made  with  the  Newton-Raphson 
algorithm  of  Haberman  (1978),  but  they  are  not  Included  here. 

LOGLINEAR  AND  LOGIT  MODELS  For  simplicity,  consider  a problem 
Involving  two  explanatory  variables  with  dimensions  I and  J , 
and  a response  variable  of  dimension  K . Thus  the  data  are  counts 
In  the  form  of  an  ixJxR  table  where  the  totals  In  the  ixj  margin 
corresponding  to  the  explanatory  variables  are  taken  as  fixed.  We 
assume  that  the  sampling  model  for  the  counts  Is  product-multinomial 
(Bishop,  Flenberg,  and  Holland,  1975).  Logit  models  (involving  K-1 
simultaneous  logit  equations)  for  this  problem  are  equivalent  to 
logllnear  models  that  treat  the  three  variables  as  If  they  are  responses, 
and  that  Include  u-terms  corresponding  to  the  main  effects  and 
Interaction  for  the  explanatory  variables  (Flenberg,  1977,  Chapter  6). 
Thus  the  Iterative  proportional  fitting  (IFF)  algorithm  used  for 
logllnear  models,  can  be  used  directly  for  logit  models  In  this 
problem. 
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NEWTON'S  METHOD  Fienb«rg  and  Stawart  (1979)  hava  usad  a variant  of 
Newton's  oathod  for  analyzing  both  logllnaar  and  logit  aodals.  Thalr 
first  algorithm  treats  the  logit  model  parameters  as  logllnaar  model 
parameters,  and  adjusts  for  the  required  conditioning  at  the  end  of 
the  computation.  A second  algorithm  proceeds  by  Initlnlly  conditioning  on 
the  explanatory  variables,  and  then  using  a somewhat  different  set 
of  computations. 

These  algorithms  involve  the  construction  of  the  upper  half  of 
a pxp  weighted  cross-product  matrix,  and  take  full  advantage 
of  the  sparseneas  of  the  n^p  design  matrix  without  actually  con- 
structing It.  The  algorithms  proceed  via  Newton's  method  with 
variable  step  length,  using  a Cholesky  decomposition  with  pivoting. 

Further  details  will  be  reported  In  the  near  future. 

GLIM  The  GLIM  algorithm,  as  developed  by  Nelder  and  his  colleagues, 
is  designed  to  handle  the  product  multinomial  sampling  model  for 
binary  response  structures,  as  well  as  other  sampling  models  not 
considered  here.  For  further  details  see  Nelder  and  Wedderbum 
(1972).  In  order  to  handle  a K-level  response  variable  In  GLIM,  the 
user  needs  to  treat  it  In  an  asymmetric  fashion,  eg.  via  the  use  of 
continuation  ratios  (see  Flenberg,  1977,  Chapter  6). 

COMPARISONS  We  have  sunnarlzed  the  qualitative  aspects  of  the  four 
algorithms  In  question  In  Table  1.  Because  the  Flenberg-Stewart 
algorithms  have  opted  for  economy  of  storage  over  efficiency  of 
operation,  the  comparisons  of  storage  requirements  between  their 
algorithms  and  GLIM  Is  misleading.  In  practice  GLIM  cannot  handle 
as  large  problems  as  can  be  handled  by  Flenberg-Stewart  algorithms. 

Although  we  have  not  made  direct  comparisons  here  on  speed  of 
convergence,  we  note  that  IPF  has  linear  convergence  properties 
while  the  other  algorithms  have  quadratic  convergence.  We  expect 
that  the  special  features  In  the  Flenberg-Stewart  algorithms  should 
allow  for  convergence  at  a slightly  faster  rate  than  GLIM,  but  this 
should  not  be  a serious  difference  between  these  methods.  More 
important  Is  the  Issue  of  numerical  stability  of  the  algorithms, 
where  again  we  anticipate  the  superiority  of  the  Flenberg-Stewart 
ones. 


Whether  one  should  use  the  logit  or  loglinear  version  of  the 
Flenberg-Stewart  algorithms  depends  on  the  size  of  the  marginal 
array  corresponding  to  the  explanatory  variables.  When  this  margin 
Is  small  some  advantages  may  accrue  to  the  loglinear  approach. 
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SUMMARY  Several  algorithms  have  been  proposed  for  the  computation  of 
maximum  likelihood  estimates  for  contingency  tables.  Since  logit 
models  can  be  treated  as  special  versions  of  logllnear  models,  many 
of  these  same  techniques  can  be  used  for  logit  models  as  well.  In 
this  paper,  we  compare  In  a qualitative  fashion  Che  relative  merits 
of  (1)  the  widely-used  method  of  Iterative  proportional  fitting,  (11) 
GLIM  as  developed  by  Nelder  and  Wedderbum,  and  (111)  two  variants  of 
Newton's  method  developed  by  Flenberg  and  Stewart. 

SOMMAIRE  Plusieurs  algorlchmes  onC  ete  proposes  afln  de  calculer  les 
evaluations  de  probablllte  maximum  pour  les  tables  d'eventuallte. 
Pulsque  les  modeles  "logit"  pouvent  etre  traltes  comme  un  cas  parti- 
culler  des  modeles  logarithmlques  et  llnealres,  la  plupart  des  tech- 
niques qul  s'appllquent  a ces  demlers  peuvent  s'appliquer  dont  aussl 
blen  aux  modeles  "logit".  Dana  cet  article,  nous  comparons  de  facon 
qualitative  les  merits  relatifs  des  methodes  sulvantes : (1)  la  methods 
frequemment  utilises  de  I'sjustement  proportlonnel  Iteratif,  (11)  la 
methods  GLIM,  developpee  par  Nelder  et  Wedderbum,  et  (111)  la  methods 
des  deux  variants  de  Newton,  developpee  par  Flenberg  et  Stewart. 


Table  1:  General  Coaparison  of  Methods. 

(1)  (11)  (111)  (Iv) 

Flenberg-Stewart  Flenberg-Scewart  GLIM  IFF 
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SSP  aatrlx  la  for  all  logllnear  parameters 
SSP  matrix  Is  for  logit  parameters  only. 
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In  this  paper  we  compare  three  different  computational  approaches 
for  maximum  likelihood  estimation  In  logit  situations: 

(a)  Iterative  proportional  fitting, 

(ij)  Iteratively  rewelghted  least  squares  as  Implemented  In  GLIM 
I (see  Welder -and -Wed Jeabum,  19^7, 

(c)  a variant  of  Newton's  methodj  as  developed  by  Flenberg  and 
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